Gesture Dynamics: Features Sensitive to Task Difficulty and Correlated with Physiological Sensors

This paper presents preliminary results regarding which features of pen-based gesture input are sensitive to cognitive stress when manipulated via changes in task difficulty. We conducted a laboratory study in which participants performed a vigilanceoriented continuous attention and visual search task. Responses to the search stimuli were entered via pen gestures (e.g., drawing a letter corresponding to the stimulus). Task difficulty was increased during predefined intervals. Participants’ input behaviors were logged, allowing for analysis of gesture input patterns for features sensitive to changes in task difficulty. We also collected physiological sensor readings (e.g., skin temperature, pulse rate, and respiration rate). Input behavior features such as gesture size and pen pressure were not affected by task difficulty, but gesture duration and length were affected. Task difficulty also affected physiological sensors, notably pulse rate. Results indicate that both gesture dynamics and physiological sensors can be used to detect changes in difficulty-induced stress.